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" ABSTRACT . , - • <> 

This -paper discusses several instructional issuss, 
defines the benefits of algorithmic prescriptions, and examines theic 
effectiveness and efficiency. An experiinent?il study asked 77 
undergraduates to master and utilize one of three forms of an 
algorithm: a rose discourse, a flowchart procedure^ or a 
mastery-inducing flowchart technique. Also examined were the ef facts 
of tlie presence or absence of the algorithm in' subsequent problem 
solving and the learners' ability to retain the algorithmic content 
over a -period of time. Both test scores and times were subjected to 
analyses of variar.ce. The results provided strong data for the 
overall effectiveness of algorithms, showing from 52 percent to over 
80 percent accuracy following an average of less than 15 minutes of 

' instruction- and practice. Rlmcst no decrement (only one percent) in 
performance was observed after a one week delay. Subjects did perform 

^.better when allowed continued use of the algorithmic representation, 
aud the flowchart group was much more efficient than the other 
groups. The interactions suqaested differential desirability of 
instructional strategy depend., no on the training needs. 
(Author/JEG) 
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Algorithms in Educationr ' Some Empirical Considerations^ 
Richard F. Schmld and Vernon S. Gerlach 
Arizona State University 

An instructional designer who relies heavily on the use of algorithms in a 
training program may be doing so preiTiaturely . The state-of-the-art regarding 
algorithms as instructional devices is that we "think" they work and that 
algorithmic instruction appears well worth very close examination. Beyond that, 
however, there is apparently no data which provides answers to the major ques- 
tions of good instructional systems design. In the following paper, we 
respond to some of these questions by identifying some of the problems pertinent 
to algorithms, we provide data for testing several important hypotheses con- 
cerning algorithmic instruction, and offer suggestions as to how and where the 
concept of algorithmic instruction might advance. 

w 

The first aim of this study was to establish th6 effectiveness of an 
algorithmic approach to the teaching of mathematical/conceptual problems. By 
comparing the effectiveness of some new type of instruction with a traditional 
method, researchers usually are able to demonstrate that the experimental 
group (e.g., some innovative technique) performed "significantly better" than 
the control. The question arises, however, "Better than what?" If a standard 
method of teaching produces 25% mastery, a new approach yielding 35% mastery 
indicates a marked improvement.. However, when emphasis is placed on criterion 
referenced training, even an luproved level of performance is unacceptable if 
its ceiling is only 35% mastery. Tnerefore, the following study avoids the 
traditional and well-established procedure of comparing an innovative method 
with an already poorly ranked alternative. Rather, mastery of a specific 
learning process is observed, and absolute, not relative, levels of perform- 
ance are assessed. 

A second critically important issue in instructional systems design is 
the extent to which the learners can retain information. The amount of ef- 
fort put into a learning task is usually directly proportional to the 
amount retained, given that all other factor.^ are held constant (Jenkins, 
Note 1). Tlie intent of any instructional innovation is to make learning 
"easier" without Iosf. of speed, accuracy, or quality of learning and/or 
performance. Tlierefore, "easier" must be carefully defined. First, extensive 
research ns well as intuition tells us that the lesson content must be some- 
what familiar to the learner and that the instructional input must be 
meaningful (Ausubel, , 1068; Haviland & Clark, 1974; Schmid, Note 2). That is, 
the instruction must be sufficiently elementary. Second, instruction muse 
contain only that information which is relevant to mastery of the task 
(Hrecko, Cerlach, & Schmid, 1976). Irrelevant or nonessential information 
loads to confusion and processing overload. That is, instruction must lead the 
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learner unambiguously to the correct result. Finally, instruction should 
always be applicable to Sn array of interrelated probleirs. The interrelation 
is best defined as the problem context, which consists of a small set of 
cognitive and psychomotor skills (Gagne' , 1965) and the environmental input. 
Processes of transfer, generalization, and learning set are products of these 
kinds of gencfral psychological schemata. Some educators fear that algorithmic 
instruction is so procedurnllzed that its use will reduce the effectiveness of 
a training system by impeding the development of such processes as transfer. 
Retention, therefore, depends upon the learner's prior knowledge, the clarity 
and precision. of the instruction, and the contribution any given unit of in- 
formation will provide for overall learning. 

The third area of interest explored in this study Is the problem of 
instructional technique. vActually using algorithms in instruction constitutes 
an upper level concern of instructional technique. How algorithms are pre- 
sented to the student represents a lower, more pragmatic level. Naturally, 
an adequate test of the algorithmic approach can be made only wheu several 
potentially effective methods of algorithmic teaching have 'been developed. 
In the present study, three standard classroom techniques were employed to 
teach an algorithm for solving tax problems. In this way, a wider range of 
instructional sampling was implemented for evaluating the effectiveness of 
algorithms in general. 

Lastly > v;e were interested in the extent to v/hich learners depend upon 
the physical presence of the algorithmic prescription. Because algorithms 
have been used for years in the form of job performance aids (e.g., recipes; 
instructions for using telephones, vending machines), there is a question 
regarding their applicability beyond providing instructions not necessarily 
committed to memory (Landa, 1974, 1968). The folj-owing study examines this 
factor for both immediate and delayed performance. It may be that algorithms 
are extremely effective and efficient when readily available, but difficult 
to internalize or retain as a systematic solution procedure, 

MoLhod • ^ 

Design and subjects . Three factors. Tax Law Availability, Instructional 
Representation, and Test Interval were combined factorially to form six 
treatment groups. Test Interval was varied as a within-subject factor. Tlie 
design was thus a 2 Availability (with tax law vs. without) X 3 Representation 
(prose vs. flowchart vs. faded flowchart) X 2 Test (immediate vs. delay). 
Analyses cff variance, with repeated measures on Test interv^al, were employed. 

Subjects; conj^lsted of 77 imdergraduate volunteers 'from Arizona State 
University. Eleven subjects were dropped from the experim»ant for failure to 
follow procedures, leaving 11 per factorial cell. Subjects were run in groups 
during normal classroom sessions. Materials were prepared beforehand and 
shuffled, so that assignment to treatment^ was completely randomized. 



Materials , Tlie instructional ta^k was adapted from Horabin's Algorithias 
(1974) • 'Hie task consisted of generating the solution to tax problems in- 
volving the purchase and sale of shares of stock. The authors created two 
sample problems for ^ach of six possible solutions, from which two/^ts of 
probiems v;ere created. Instruction on how to apply the tax law tfool^ one of 
three representational forms; the prose condition" received a venial descrip- 
tion of the law, which followed an if /then format; the flowchart condition 
received the same Information in a flowchart' form, such that each decision 
(dlscrininator) was binary, leading to another discriminator, until the termi- 
nal solution box (operator) was reached; the faded flowchart was exactly the 
same as the flowchart treatment, except that for each problem completed, one 
of the discriminators necessary for the solution of subsequent problems was 
deleted. The deleted information was made available only if the subject was 
unable to recall it. Corrective feedback was supplied in separate boolclets. 
The q^uestionnaire following the treatment asked about the learner^ s strategies 
and t'eactions to the instruction* vThe posttest contained six additional 
randomly ordered problems of the same type, without feedback. The delayed 
posttest ahd the immediate posttest , were identical, except for the order of 
the problems. ' » 

Procedure ./ The experimental sequence consit^ted of (a) an orientation, 
(b) a practice 'session, (c) ah immediate posttestj^ (d) a questionnaire, 
(e) a one-week delay posttest, and (f) a questionnaire. In addition to com- 
pleting six practice and six posttest problems, subjects recorded the time 
cin spaces provided at. the start and finish of each problem. Following the 
introduction read aloud by . the experimenter^ subjects completed all phases at 
their own pace. During both posttests, only half the subjects from each con- 
dition were supplied the tax law for solving the problems. 



Results 


Achi rvome jn t 

Ail protocols were scored for number correct, with one point for the 
dollar amount and one for the tax status (charged, allowed, or no tax). 
Omissions were counted as errors. 

0 

lliree Representation x two Availability analyses of variance were con- 
ductud on the practice and immediate* posttest sessions. A repeated measures 
anova using the delayed test data was then performed. Tae repeated measures 
anova was performed twice, (a) using groups as represented by the with/without 
distinction on che iimnedlate posttest, and (b) using groups separated according 
to the Availability factor on the delayed test. 

. As expected, no differences were found between groups during the 
practice session. However, posttest performance was significant for Repre- 
sentation, F (2, 60) « 3.20, £ < .04, r.nd Availability, F (1, 60) ^ 21.82, 
£ < .001. Sche:tfe^ tests on the Representation factor ordered the groups: 
proiie « flowchart > faded flowchart. 



The repeated measures anova on original g^roups yielded significance for 
the Availability main effect, F (1, 54) = 16.83, £ < .001, and for the 
Availability x Teat interaction, F (1, 54) = 8.17, £ < .006. Not surpris- 
.Ingly, subjects allowed to use the procedure during the test session performed 
slgnif i-antly better than the group from whom the -{jrocedure was withheld. Of 
greater interest, however, was the interaction. Individual comparisons 
showed that learners who were allowed to use the procedure during initial 
testing performed significsmtly worse on the delayed test, due to the fact 
that the procedure vas v;ithhe*ld from half the subjects during the delay. On 
the other hand, significant increment in performance was noted for subjects 
not allowed to use the' procedure during initial testing, ah effect likely 
attributable to the renewed availability of the procedure for half those subject 

The repeated measures anova utilizing a regrouping of subjects on the 
delay according to procedure availability yielded significance for the Availa- 
bility main effect, F (1, 54) = 64.77, £ < .001, and tlje Representation x 
Availability interaction, F (2, 54) = 3.37, £ < .04. Subjects using the pro- 
cedure produced higher scores than those without. The significant interaction 
demons tra°ted that>yhen subjects are supplied v/ith the procedure,' the fla^chart 
is the most effecliive representation for solving the problems. However, when 
the procedure is removed, the fading group fell significantly below the prose 
and flowchart groups, V/hich did not differ. These effects reiterate the 
ineffectiveness of the fading instructional strategy, especially emphasized by 
the fact that tl)e flowchart and faded flowchart groups did not "differ in any 
way following the practice portion of the experiment. This interactioh is 
shown in Figure 1. 

Time 



Time data were generated by computing the mean number of laeconds taken 
per problem solution. Omitted problems were not included in the estimates. 

Three Representation x two Availability analyses of variance were conduc- 
ted on the practice and immediate posttest sessions. A repeated measures 
anova using the delayed ,test data was then performed. As with the test scores 
the repeated measures anova was performed t\>^ice, (a) using the immediate post-* 
test Availability distinction, and (b) using the delayed posttest Availability 
regroup Lng. ' 

Times taken during the practice session differed significantly, F (2, 60) = 
8.44, £ < .001, with Scheffe' tests ordering the meano: fWchart < prose = 
faded flowchart, lliese initial differences, however, disappeared during the 
immediate posttest. 

The repeated measures anova on the original groups yielded dif 'erences for 
for the Representation main effe.n, F (2, 54) = 3.74, £ < .03, and the within 
subject rest factor, F (1, 54) - 12.46^ £ < .001. Scheffe' tests ordered the 
Representation means: flowchart « prose < faded flowchart. More time was 
spent on the inmiediate posttest than the delayed test. 
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Figure 1. Representation -x availability interaction from the 
repeated measures anova on test scores regrouped 
on delayed test availability 
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When subjects were regrouped according to pVocedure availability on the 
delayed test, significant effects were found for the Representation factor, 

1 (2, 5'0 = 3.83, £ < .03, the test/retest factor, F (1, 54)'= fl.98, p < 
.001, and the Representation x Availability intaeactlon, F (2, 54) = 6.16i 

2 < .004. Scheffe' tests ordered the representational treatment means: 
flowchart = prose < faded flowchart. Less time was spent on the delayed 
items than the immediate test items. The interaction produced some very 
Ciirious. results; the flowchart group spent the same^ount of time either with 
or without the procedure, the prose group spent significantly less time (with- 
out, and the faded flowchart subjects spent significantly more time without 
the procedure present. Further, the flowchart representation subjects worked 
significantly faster than the prose version subjects when the proced^re was 
available, w^iereas the opposite held "true when the procedure was removed. A 
graphic representation appears In Figure 2. 



Discussion 

The results of this investigation supply convincing answers to the pri- 
mary questions raised in the rationale. Overall effectiveness of the 
algorithmic prescript lofts was quite high, ranging from 80% to 52% accuracy. 
Surprisingly, performance decreased only one percentage point after one week. 
These effects persisted in spite of the fact that 1/4 of the subjects were 
not allowed to examine the procedure at any time after the practice session, 
and 1/4 did not see the procedure after the immediate test. Nevertheless, the 
availability of the procedure during the immediate test stlill facilitated 
learning signifi^tly, in that the group receiving the procedure achieved 
superior scores on the delayed test, as indicated by the Availability x Test 
interaction. . , 

These data suggest two conclusions/ First, tlje effectiveness of ^ the 
algorithm was demonstrated by the high performance not only immediately fol-- 
.lowing instruction, but also one week later. The classic forgetting curve 
appears to have been defied, even \.ithout ihe presence of a ceiling effect. 
Tliua, the absolute value of algorithms was demonstrated, lliis effect is com- 
pounded by the hifeh ef fi ciency of instruction (n\eati learning time, approximately 
15 minutes, Wi-thln the task requirement of this study, algorithms 

wore extremely good instructional devices. . . 

«> 

The second conclusion drawn from these data is that although learners can 
indeed memorize the content of an algorithm for later use, enabling them to 
continue using the procedure is significantly more effective. \-Jh^n the 
procedure was made available, 80% performance was found on both immediate and 
delayed tests. The Availability x Test interaction graphically shows the 
potency of the presence of the algorithm. The significant convergence from 
both immediate test means to the nonsignificant difference in the delay test 
sui',ge8ts that the presence of the procedure is equally critical regardless of 
LliG subjects' counterbalancing membership. 
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The next q.uestion referred to specific instructional techniques. Although 
a pilot fitudy rlid not yield differences bet\/een the three algorithmic repre- 
sentations,- the revision in tht main study yielded some interesting and 
uiiexpecte^l results. The flo\>rcndrt and prose groups performed equally well on 
the imm-diate test, supporting-^he contentior^ that both produce equivalent 
Information and that both are equal4y algorithmic. The mode of representation 
apparently did not differentially effect processing. However, the faded flcv;- 
chart group, contra^ry to expectations, performed significantly v;orse than the 
other groups, despite the fact that its instruction was derived from the 
identical information base. 'The special processing demands "^required of the. 
fading condition apparently interfered with task^ acquisition , as indicated by " 
the slgnifi'cant performance decrement in the Representation x Availability 
interaction. These differences were especially strong v;hen the procedure .was 
^withheld. Thus, the technique developed to increase learning actually hindpred 
Qverall performance. One possible explanation for these results is that the 
procedure placed undue emphasis %ti the disoriminators , or decision-making 
aspect, and too little on the operator?, which actually fonried the solutions. 
It may also be that this strategy requires more practice than the other treat- 
ments (Schmid &^Cerlach, Note 3). A tradeoff with efficiency may yield b'e'tter 
effectiveness. 

? * 

In an overall assessment of efficiency, we noted not only the short 
period of ins trudtion' required, but also that efficiency for the algorithmic* 
approach Improved 28% over the time delay. While any recommendations to 
teachers or instructional developers must be qualified, these results 'suggest 
that the best technique for task performance would be to supply the learner 
with a flowchart which remains available for reference. 

In summary, algorithms were found to be both effective and efficient in 
absolute, rather than comparative, terms. They seem especially resistant to 
forgetting. Differences between the flowchart and prose treatments remain 
somewhat unclear, each having shown distinct advantages depending upon pro- 
cedure availability and the retention interval. Our present conclusion is that 
the same underlying learning processes are occurring regardless of the repre- 
sentation-. V'e are now completing several studies which are examining the 
strudtural and cognitive characteristics of algorithms by means of cybernetics 
antl structural learning theory. Finally, forcing the processing of a flow-" 
chart In the manner ^employed here is detrimental to both the effectiveness 
ami efficiency of algorithmic instruction. Further experimentation must 
distinguish between the mathemagenic and conceptual acquisition effects 
responsible fqr these data. 
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